Welcome

Welcome to EPsy 8264: Advanced Multiple Regression Analysis. This is an advanced seminar for doctoral students in education. The pre-requisites for this course are EPsy 8251 and EPsy 8252.


Instructor

Andrew Zieffler (zief0002@umn.edu)
Virtual Office: Zoom
Office Hours: Wednesday 9:00 AM–10:00 AM; and by appointment

Teaching Assistant

Vimal Rao (rao00013@umn.edu)
Virtual Office: Zoom
Office Hours: Tuesday 5:30 PM–6:30 PM; and by appointment


Classroom

  • Tuesday/Thursday (9:45–11:00): Zoom


Syllabus

  • The course syllabus is available here.


Textbooks

The course textbooks are available via the University of Minnesota library (Fox) and online (Grolemund & Wickham).


Prerequisites

Prerequisite knowledge include topics from a basic statistics course:

  • Foundational topics in data analysis;
    • Design (e.g., random assignment and random sampling)
    • Descriptive statistics and plots
    • One- and two-sample tests

And, topics from EPsy 8251: Methods in Data Analysis for Educational Research I:

  • Statistical Computation
    • Using R
    • Data wrangling/manipulation
    • Plotting
  • Correlation;
  • Simple regression analysis;
    • Model-level and coefficient-level interpretation
    • Ordinary least squares estimation
    • Standardized regression
    • Partitioning sums of squares
    • Model-level and coefficient-level inference
    • Assumption checking/residual analysis
  • Multiple linear regression
    • Model-level and coefficient-level interpretation and inference
    • Assumption checking/residual analysis
    • Working with categorical predictors (including adjusting p-values for multiple tests)
    • Interaction effects

And topics from EPsy 8252: Methods in Data Analysis for Educational Research II:

  • Dealing with nonlinearity;
    • Quadratic effects
    • Log-transformations
  • Probability distributions;
    • Probability density
  • Maximum likelihood estimation;
  • Model selection;
    • Information criteria
  • Linear mixed-effects models (cross-sectional/longitudinal)
    • Basic ideas of mixed-effects models
    • Fitting models with random-intercepts and random-slopes
    • Assumptions
    • Likelihood ratio tests
  • Generalized linear models
    • Logistic models

For the topics listed, students would be expected to be able to carry out an appropriate data analysis and properly interpret the results. It is also assumed that everyone enrolled in the course has some familiarity with using R. If you need a refresher on any of these topics, see:


Calendar

Below is the tentative schedule for the class. The dates are subject to change at the instructor’s discretion.


Date Topic
  Sept. 08 Welcome to EPsy 8264
Unit 01: Mathematics
  Sept. 10 Some Mathematics Relevant to Regression
  Sept. 15
  Sept. 17 A Bit of Linear Algebra
  Sept. 22
  Sept. 24 Regression using Matrices
  Sept. 29
Unit 02: Regression Diagnostics
  Oct. 01 Regression Diagnostics
  Oct. 06
Unit 03: Dealing with Heteroskedasticity
  Oct. 08 Variance Stabilizing Transformations
  Oct. 13 Weighted Least Squares (WLS) and Sandwich Estimation
Unit 04: Dealing with Collinearity
  Oct. 15 Collinearity Diagnostics
  Oct. 20 Principal Components Analysis
  Oct. 22
  Oct. 27 Biased Estimation and Shrinkage
  Oct. 29
Unit 05: Model Selection
  Nov. 03 NO CLASS— Election Day
  Nov. 05 Traditional Methods of Model Selection
  Nov. 12 Cross-Validation
  Nov. 17
Unit 06: Piecewise Modeling
  Nov. 24 Piecewise Regression
  Nov. 26 NO CLASS—Thanksgiving Break
  Dec. 01 Piecewise Regression
  Dec. 03 Spline Models
  Dec. 08
  Dec. 10
  Dec. 15 TBA

Assignments

Below are the due dates for the assignments, as well as links to the PDF file for each assignment. The due dates may change at the instructor’s discretion. Any revised due dates will be announced in class and posted to the website.


Assignment Due Date PDF
Assignment #1: Regression and Mathematics Sept. 24
Assignment #2: Matrix Algebra for Linear Regression Oct. 06
Assignment #3: Regression Diagnostics Oct. 15
Assignment #4: Using WLS to Model Data with Outliers Oct. 22
Assignment #5: Principal Components Analysis Nov. 10
Assignment #6: Shrinkage Nov. 24
Assignment #7: Cross-Validation Dec. 08
Assignment #8: Regression Splines Dec. 15

Data

Below are the links to the data sets and data codebooks used in the notes, scripts, and assignments.


Name Data Codebook
bluegills.csv
canadian-prestige.csv
credit.csv
davis.csv
davis-corrected.csv
duff.csv
duncan.csv
education-expenditures.csv
equal-education-opportunity.csv
evaluations.csv
fake-piecewise-data.csv
galton.csv
grade_data.csv
houston.csv
loess.csv
mcycle.csv
mpls-violent-crime.csv
polynomial-example.csv
relate.csv
slid.csv
stack-1979.csv
tokyo-water-use.csv

Materials

Welcome to EPsy 8264

Prior to class:

After class:

  • If you will be using your computer for statistical computing (rather than the RStudio server), install (or update) R and RStudio onto your personal computer. See R and RStudio Installation and Setup for help.

Some Mathematics Relevant to Regression

  • Notes 01: Some Mathematics Relevant to Regression [pdf] [rmd]
  • Notes 02: Some Theory Underlying Simple Regression [pdf] [rmd]
  • Notes 03: Some More Theory Underlying Simple Regression [pdf] [rmd]
  • Notes 04: Some OLS Theory: Proofs [pdf] [rmd]

A Bit of Linear Algebra